Evaluation of bias and gender/racial concordance based on sentiment analysis of narrative evaluations of clinical clerkships using natural language processing.

There is increasing interest in understanding potential bias in medical education. We used natural language processing (NLP) to evaluate potential bias in clinical clerkship evaluations. Data from medical evaluations and administrative databases for medical students enrolled in third-year clinical clerkship rotations across two academic years. We collected demographic information of students and faculty evaluators to determine gender/racial concordance (i.e., whether the student and faculty identified with the same demographic). We used a multinomial log-linear model for final clerkship grades, using predictors such as numerical evaluation scores, gender/racial concordance, and sentiment scores of narrative evaluations using the SentimentIntensityAnalyzer tool in Python. 2037 evaluations from 198 students were analyzed. Statistical significance was defined as P < 0.05. Sentiment scores for evaluations did not vary significantly by student gender, race, or ethnicity (P = 0.88, 0.64, and 0.06, respectively). Word choices were similar across faculty and student demographic groups. Modeling showed narrative evaluation sentiment scores were not predictive of an honors grade (odds ratio [OR] 1.23, P = 0.58). Numerical evaluation average (OR 1.45, P < 0.001) and gender concordance between faculty and student (OR 1.32, P = 0.049) were significant predictors of receiving honors. The lack of disparities in narrative text in our study contrasts with prior findings from other institutions. Ongoing efforts include comparative analyses with other institutions to understand what institutional factors may contribute to bias. NLP enables a systematic approach for investigating bias. The insights gained from the lack of association between word choices, sentiment scores, and final grades show potential opportunities to improve feedback processes for students.

Availability of Physical Activity Tracking Data from Wearable Devices for Glaucoma Patients.

Physical activity has been found to potentially modulate glaucoma risk, but the evidence remains inconclusive. The increasing use of wearable physical activity trackers may provide longitudinal and granular data suitable to address this issue, but little is known regarding the characteristics and availability of these data sources. We performed a scoping review and query of data sources on the availability of wearable physical activity data for glaucoma patients. Literature databases (PubMed and MEDLINE) were reviewed with search terms consisting of those related to physical activity trackers and those related to glaucoma, and we evaluated results at the intersection of these two groups. Biomedical databases were also reviewed, for which we completed database queries. We identified eight data sources containing physical activity tracking data for glaucoma, with two being large national databases (UK BioBank and All of Us) and six from individual journal articles providing participant-level information. The number of glaucoma patients with physical activity tracking data available, types of glaucoma-related data, fitness devices utilized, and diversity of participants varied across all sources. Overall, there were limited analyses of these data, suggesting the need for additional research to further investigate how physical activity may alter glaucoma risk.

Concept Coverage Analysis of Ophthalmic Infections and Trauma among the Standardized Medical Terminologies SNOMED-CT, ICD-10-CM, and ICD-11.

Purpose: Widespread electronic health record adoption has generated a large volume of data and emphasized the need for standardized terminology to describe clinical concepts. Here, we undertook a systematic concept coverage analysis to determine the representation of clinical concepts in ophthalmic infection and ophthalmic trauma among standardized medical terminologies, including the Systematized Nomenclature of Medicine Clinical Terms (SNOMED-CT), the International Classification of Diseases (ICD) version 10 with clinical modifications (ICD-10-CM), and ICD version 11 (ICD-11). Design: Extraction of concepts related to ophthalmic infection and ophthalmic trauma and structured search in terminology browsers. Data Sources: The American Academy of Ophthalmology Basic and Clinical Science Course (BCSC), SNOMED-CT, and ICD-10-CM terminologies from the Observational Health Data Sciences and Informatics Athena browser, and the ICD-11 terminology browser. Methods: Concepts pertaining to ophthalmic infection and ophthalmic trauma were extracted from the 2022 BCSC free text and index terms. We searched terminology browsers to identify corresponding codes and classified the extent of semantic alignment as equal, wide, narrow, or unmatched in each terminology. The overlap of equal concepts in each terminology was represented in a Venn diagram. Main Outcome Measures: Proportions of clinical concepts with corresponding codes at various levels of semantic alignment. Results: A total of 443 concepts were identified: 304 concepts related to ophthalmic infection and 139 concepts related to ophthalmic trauma. The SNOMED-CT had the highest proportion of equal coverage, with 82.0% (249 of 304) among concepts related to ophthalmic infection and 82.0% (115 of 139) among concepts related to ophthalmic trauma. Across all concepts, 28% (124 of 443) were classified as equal in ICD-10-CM and 52.8% (234 of 443) were classified as equal in ICD-11. Conclusions: The SNOMED-CT had significantly better semantic alignment than ICD-10-CM and ICD-11 for ophthalmic infections and ophthalmic trauma. This demonstrates opportunity for continuing advancement of representation of ophthalmic concepts in standardized medical terminologies.

Assessing Usability of Smartwatch Digital Health Devices for Home Blood Pressure Monitoring among Glaucoma Patients.

Glaucoma is a leading cause of blindness worldwide. Blood pressure (BP) dysregulation is a known risk factor, and home-based BP monitoring is increasingly used, but the usability of digital health devices to measure BP among glaucoma patients is not well studied. There may be particular usability challenges among this group, given that glaucoma disproportionately affects the elderly and can cause visual impairment. Therefore, the goal of this mixed-methods study was to assess the usability of a smart watch digital health device for home BP monitoring among glaucoma patients. Adult participants were recruited and given a smartwatch blood pressure monitor for at-home use. The eHEALS questionnaire was used to determine baseline digital health literacy. After a week of use, participants assessed the usability of the BP monitor and related mobile app using the Post-study System Usability Questionnaire (PSSUQ) and the System Usability Scale (SUS), standardized instruments to measure usability in health information technology interventions. Variations in scores were evaluated using ANOVA and open-ended responses about participants' experience were analyzed thematically. Overall, usability scores corresponded to the 80th-84th percentile, although older patients endorsed significantly worse usability based on quantitative scores and additionally provided qualitative feedback describing some difficulty using the device. Usability for older patients should be considered in the design of digital health devices for glaucoma given their disproportionate burden of disease and challenges in navigating digital health technologies, although the overall high usability scores for the device demonstrates promise for future clinical applications in glaucoma risk stratification.